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This  report  is  on  a  pilot  study  to  demonstrate  some  of  the  capabilities  of  remote  sensing  in 
intelligence  gathering.  A  wide  variety  of  issues,  both  quantitative  and  qualitative  ,are 
addressed.  SPOT  satellite  images  were  used  for  object  classification  using  statistical  and 
neural  net  techniques,  object  height  determination  and  image  resolution  enhancement.  A 
new  procedure  was  developed  for  detecting  small  and  man-made  objects  in  multispectral 
or  colour  images.The  procedure  will  be  presented  in  a  separate  report. 
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I  INTRODUCTION  AND  OBJECTIVES 


Surveillance  and  intelligertce  collection  have  been  given  the  highest  priority,  particularly 
at  a  time  of  reduced  likelihood  of  hostility  (White  Paper  ,1987).  Traditionally  the 
intelligence  organisations  have  heavily  relied  on  high  resolution  images,  mostly  from 
restricted  sources.  Most  of  the  high  resolution  images  are  black  and  white  images,  taken  in  a 
wide  spectral  band.  Processing  of  high  resolution  images  is  very  time-intensive  aiul 
expensive.  The  groutKl  resolution  of  commercial  images  are  coarser  than  that  of  the  military 
satellite  images.  However,  this  disadvantage  can  be  often  offset  by  the  availability  of  data 
in  more  than  one  spectral  band  and  for  a  wider  area.  It  is  important  to  weigh  the  pros-cons  of 
using  any  type  of  image  data . 

In  the  past,  since  the  launching  of  the  first  commercial  intaging  satellite  21  years  ago,  the 
importance  of  commercial  images  in  surveillance  has  been  greatly  ignored.  Most  of  the 
capability  studies  in  the  past  (]oye,l991;  Richelson,1990)  have  concluded  that  the 
applications  of  commercial  satellite  images  in  surveillance  are  very  limited.  Table  I.l  in 
Appendix  I  summarises  this  pessimistic  view. 

However,  towards  the  latter  part  of  1980s  many  ittdependent  investigators  found  SPOT 
pictures  very  useful  in  investigating  military  targets-  In  order  to  find  out  what  can  be  seen 
from  commercial  satellites  in  space  ,  the  Carnegie  Endowment  initiated  a  series  of  studies  in 
which  SPOT  and  Soyuzkarta  KFA-1000  images  were  analysed  by  the  professionals  in  the 
field.  Their  results  were  quite  surprisingly  different  from  the  earlier  pessimistic  results. 
Table  1.2  in  Appendix  I  summarises  their  results  (Zimmerman,  1990).  The  Carnegie 
Endowment  study  revealed  that  what  images  reveal  very  much  depends  on  the  expertise  of 
the  analyser.  As  such,  the  processing  of  the  images  is  a  crucial  step  in  deciding  whether  a  set 
of  images  is  useful  or  not 

In  addition,  during  the  1991  Gulf  war  both  Landsat  and  SPOT  images  were  extensively  and 
effectively  used  (Shettigara  V.K.,1992).  These  aspects  prompted  us  to  have  a  fresh  look  at 
the  capabilities  of  commercial  satellite  images.  A  comprehensive  discussion  on  the 
potential  of  commercial  satellite  images  in  wide  area  surveillance  is  available  in  Gale 
(1992).  In  this  report  we  present  both  quantitative  and  qualitative  image  processing  results  to 
demonstrate  that  the  commercial  satellite  images  are  capable  of  providing  much  more 
information  than  previously  believed. 

The  task  '  Intelligence  and  commercial  satellite  images  ‘  was  a  pilot  study  to  evaluate  and 
demonstrate  some  of  the  capabilities  of  commercial  satellite  images  for  intelligence 
gathering  purposes.  The  task  was  to  investigate  the  capabilities  and  limitations  of  the 
commercially  available  multi-sensor  and  multi-band  images  in  detecting  and  identifying 
civilian  artd  engineering  objects.  The  task  was  expected  to  provide  the  basis  for  comparing  the 
perfbrmartces  of  some  of  the  traditional  and  alternate  sources  of  intelligence  and  mapping  in 
the  future. 
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As  listed  in  the  contents,  the  images  were  used  for  feature  extraction  using  statistical  aiKl 
neural  net  techniques,  tree  height  determination  and  image  resolution  erdtancement.  A 
seperate  report  is  available  on  a  new  procedure  to  detect  man-made  and  small  objects  in 
multispectral  colour  images. 


2  ROLE  OF  REMOTE  SENSING  IN  DEFENCE 


The  term  '  Remote  Sensing'  is  used  in  this  report  to  indicate  the  study  of  images  from  civilian 
satellites.  No  regular  use  is  made  of  remote  sensing  data  in  defence,  particularly  in  Australia, 
for  the  following  reasons:  1.  From  the  defence  point  view  the  civilian  satellites  lack  the 
spatial  resolution  of  military  satellites.  The  best  resolution  available  is  10m.  in  SPOT 
digital  images  and  15  m  in  synthetic  aperture  radar  images  from  Almaz  satellite.  A 
resoluhon  of  5-7  m  is  claimed  for  scaimed  images  produced  from  the  Russian  KFAIOOO  camera. 
2.  The  control  of  data  rests  in  outside  civilian  agencies  and  its  availability  may  not  be  relied 
upon  at  the  time  of  need  3.  the  revisit  time  is  too  far  apart  for  effective  recoitnaissance  and 
surveillance  efforts. 

It  IS  reasonable  to  accept  that  at  present  commercial  satellite  images  do  not  meet  some 
defence  needs,  particularly  at  the  high  resolution  end.  However,  it  is  reported  recently 
(EOM,1992)  that  Russian  digital  images  with  a  resolution  of  2*3  m  are  now  available  In  the 
open  market.  As  such  there  is  no  si  :igle  source  of  imagery  that  could  satisfy  all  the  needs  of 
defence.  Many  issues  in,  for  example,  terrain  intelligence  collection  require  neither  very  high 
resolution  nor  frequent  revisit.  To  detect  activities  such  as  road  construction,  site  clearance  for 
building  construction  and  runway  construction  the  remote  sensing  data  are  adequate  and 
probably  better  because  of  the  multispectral  nature.  Another  major  application  of  remote 
sensing  is  in  change  detection  facilitated  by  the  availability  of  multiple  data  sources  with  a 
good  frequency  of  coverage.  The  moderate  spatial  resolution  of  civilian  satellites  are 
advantageous  in  some  terrain  analysis  applications  due  to  the  wide  area  that  a  single  image 
can  cover.  The  processing  time  increases  as  the  inverse  square  of  resolution.  It  will  be  wasteful 
to  use  higher  resolution  data  when  the  problem  can  be  solved  with  lower  resolution  images. 
Another  attraction  of  remote  sensing  is  the  multispectral  character  of  the  data  providing 
spectral  resolution  in  addition  to  the  spatial  resolution.  The  disadvantages  of  lower  spatial 
resolution  can  be  offset  to  some  extent  by  the  higher  spectral  resolution.  Finally  the 
availability  of  ever  improving,  sophisticated  but  cheap  data  processing  capabilities  in  micro 
computers  and  work  stations  make  the  remote  sensing  technology  very  attractive. 

The  1991  Gulf  war  clearly  demonstrated  the  role  of  remote  sensing,  in  conjunction  with  other 
sources  of  information,  at  bodi  the  tactical  and  the  strategic  levels.  SPOT  and  Landsat-TM 
images  were  exteixsively  used  for  three  nukjor  purposes  during  the  Gulf  war;  1.  to  prepare  up  to 
date  maps  of  the  area,  2.  as  input  to  'automated  mission  planning  systems'  (Bemard,1991). 
and  3.  for  strategic  reconnaissance,  particularly  using  SPOT  images.  The  Gulf  war  narrowed 
the  technological  gap  between  the  military  and  the  civilian  technology.  (Anson  and 
Cummings,  1991).  The  work  presented  in  this  report  will  further  illustrate  the  capabilities 
that  are  relevant  to  defence. 
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3  THE  STUDY  AREA  AND  THE  DATA  BASE 

3.1  The  study  area 

The  Defence  Science  and  Technology  Organisation  (DSTO)  area  in  Salisbury,  about  23  km 
north  of  Adelaide,  was  selected  for  the  study.  The  site  is  located  around  138*37'  E  34*43'  S. 
The  area  was  chosen  for  the  following  reasons: 

1 .  Two  sets  of  SPOT  satellite  images,  an  airborne  SAR  image  and  digital  map  data  were 
available  for  the  area. 

2.  The  area  has  a  mixture  of  defence  infrastructures,  an  industrial  complex  and  civilian 
suburbs.  The  study  area  is  maiidy  the  DSTO  premises  and  Edinburgh  RAAF  base, 
surrounded  by  the  residential  suburbs  of  Salisbury  aiul  Elizabeth,  and  GMH  Holden 
industrial  estate.  The  area  has  an  extensive  network  of  roads  of  various  kinds, 
railway  tracks,  and  numerous  buildings  of  different  types  and  dimensions. 
Construction  activities  are  fairly  frequent  in  the  area. 

3.  The  northern  half  of  the  area  is  bound  by  open  fields  and  farm  lands. 

4.  One  of  the  important  requirements  of  remote  sensing  studies  is  the  need  to  collect 
ground  truth  data.  From  this  point  of  view  the  area  was  readily  accessible  for 
research  purposes. 

The  terrain  is  flat.  The  eastern  portion  of  the  area  has  been  planted  with  rows  of  trees  that 
run  roughly  north-east  and  north-west  The  rows  consist  of  either  aleppo  pines  or  sugar  gums. 
At  places  both  types  of  rows  are  seen  side  by  side ,  with  usually  the  aleppo  pine  rows  in  the 
south  and  the  sugar  gum  rows  in  the  north.  The  rows  of  trees  surround  open  fields  and  factory 
like  buildings.  Most  buildings  have  asbestos  saw-tooth  roof  with  soudt  lighting.  A  few  have 
tin  roofs. 

The  Edinburgh  Air  base  on  the  north  western  part  of  the  area  has  a  sealed  runway  in  the  N-S 
direction,  taxi  ways  and  apron  areas  for  aircraft  parking.  There  is  a  disused  unsealed  runway 
in  the  NE  direction.  The  major  buildings  are  near  the  hangars  adjoirung  the  apron,  to  the  east 
of  dw  main  runway.  The  administrative  buildings  of  the  air  base  and  the  residential  quarters 
are  situated  to  the  east  of  the  airfield.  An  ammunition  storage  facility  exists  near  the 
runway. 

3.2  The  data  base 

One  multispectral  atvi  two  SPOT  panchrorrtatic  images  were  utilised  in  this  study  along  with 
two  aerial  photographs.  No  image  processing  has  been  doite  on  the  photographs  as  they  were 
used  only  for  reference.  The  first  panchromatic  image  and  the  multispectral  image  were  taken 
on  21st  April  1987  where  as  the  second  panchromatic  image  was  taken  on  25  April  1987.  The 
panchromatic  images  are  off  nadir  looking  and  form  a  stereo  pair.  The  image  parameters  are 
summarised  in  Table  3.1. 
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TKe  first  aerial  photograph  was  taken  on  5th  January  1987  and  the  second  was  taken  on 
3rd  October  1990.  The  scanned  aerial  photo  of  Sth  January  is  shown  in  Figure  3.1,  with  a  pixel 
resolution  of  1.6  m.  This  aerial  photograph  was  taken  109  days  before  the  satellite  images 
were  acquired.  The  terrain  conditions  as  seen  in  the  photograph,  particularly  with  regard  to 
vegetation,  may  not  be  the  ambient  condition  when  the  satellite  images  were  collected. 

The  Panchromatic  image  of  the  25th  April  (not  shown)  and  the  Multispectral  image 
(Figure  3.2)  of  the  21st  April  were  registered  on  to  the  Panchromatic  image  of  the  21st  April 
(Figure  3.3)  utilising  nearest  neighbour  sampling  and  first  order  polynomial  transformation. 
This  involved  resampling  the  Multispectral  image  from  twenty  metre  pixels  to  ten  metre 
pixels.  All  the  processing  was  done  on  this  set  of  images. 


Table  3.1  Imaging  Parameters 


Satellite: 

SPOTl 

SPOTl 

Date: 

21/4/87 

25/4/87 

Local  Time 

10. 06. 05. 

10.  29. 18. 

Latitude: 

S  34°  53' 39* 

S  34°  53 '39" 

Longitude: 

E  138°  33' 32" 

E  138°  35'  13" 

Orientation: 

12.9° 

9.2° 

IncideiKe: 

R23.4° 

L15.5° 

Azimuth: 

39.2° 

32.1° 

Elevation 

345° 

36.2° 

In  Figure  3.4  is  a  part  of  the  existing  CIS  vector  data  base  imported  from  the  Australian 
Surveying  and  Land  Information  Group  (AUSUG).  The  accuracy  of  the  data  is  about  1  m.  This 
vector  based  data  can  be  overlain  on  the  raster  images  and  information  extracted  from  the 
images  can  be  added  to  the  vector  data  base.  The  information  collected  and  stored  in  the 
vector  data  base  can  annotated  and  displayed  with  ease. 

The  data  processing  in  this  task  was  done  on  an  Intergraph  workstation  using  ISI-2  image 
processing  software  and  locally  developed  computer  programs. 
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Figure  3.1  Scanned  colour  aerial  photograph  of  the  study  areataken  on  S  Jan  1987. 
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Figure  3-J  Pancromatic  SPOT  image  of  the  areataken  on  21  April  1987.  Pixel  resolution  10  m 
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4  PATTERN  RECOGNITION 


The  term  pattern  recognition  is  used  to  mean  various  aspects  in  image  processing  literature. 
Pattern  recognition  is  a  classification  problem,  including  clustering  and  supervised 
classification  (Young  and  Calvert,  1974).  Prior  to  classification,  however,  a  detection  stage 
may  be  involved  depending  on  the  technique  used.  Sometimes  the  term  'pattern  recognition'  is 
stretched  to  include  image  enhancement.  Some  authors  constrain  it  to  mean  'innage 
understanding'  that  is  mainly  concerned  with  representing  real  world  objects  and  managing 
them  in  a  computational  environment  (Sleigh,1983).  Feature  extraction  is  another  term 
frequently  used  to  mean  pattern  recognition.  However,  the  use  of  the  term  'feature  extraction' 
will  be  avoided  in  this  report  as  the  term  'feature'  is  some  times  used  to  mean  the  bands  in  the 
image  arui  feature  extraction  in  such  a  context  refers  to  the  selection  of  features  (bands)  or  the 
reduction  of  the  dimensiortality  of  the  data  set  without  significantly  losing  the  information 
content.  In  this  report,  'Pattern  recognition'  is  used  to  mean  the  classification  of  the  images  to 
objects  of  known  physical  descriptiott. 

4.1  Pattern  recognition  -  a  brief  review 

There  are  three  main  approaches  to  pattern  recognition; 

(a)  statistical  pattern  recognition, 

(b)  optical  and  digital  pattern  correlation  (template  matching),  and 

(c)  model-based  vision.  A  good  review  of  these  techniques  is  available  in  Mundy  (1991). 

Statistical  pattern  recognition  can  be  implemented  using  Bayesian  classifiers,  clustering 
procedures  or  more  recently  neural-net  classifiers.  The  strength  of  statistical  pattern 
recognition  lies  in  its  ability  to  process  multi-dimaisional  data  in  a  multi-dimensional  space 
without  knowing  the  spatial  structure  of  the  data.  However,  with  some  difficulty  the 
spatial  structure  of  the  data  may  be  iiKorporated  in  the  statistical  procedures  for  a  better 
result.  Other  techniques  described  below  process  variates  of  the  multivariate  data 
separately. 

In  the  technique  of  pattern  correlation,  an  instance  of  the  object  is  taken  as  a  template  and  its 
correlation  with  different  parts  of  the  input  image  is  determined.  The  targets  are  determined 
based  on  maximum  correlation  achieved.  Correlation  techniques  can  be  made  translation, 
rotation  and  scale  invariant.  However,  correlation  is  not  intensity  invariant.  Variation  in 
shadows  can  give  false  results  (Mundy,1991). 

In  model  based  pattern  recognition  procedures  die  image  is  initially  processed  to  extract  a  2D 
description  of  the  objects.  Then,  using  a  3D  model,  a  2D  representation  is  predicted.  The 
classification  is  achieved  based  on  the  prediction  accuracy.  The  advantage  of  tihis  technique 
is  that  it  is  intensity  invariant.  However,  the  technique  suffers  as  the  pixel  resolution 
becomes  coarser.  This  limits  its  application  in  pattern  recognition  in  civilian  satellite  images. 
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From  the  above  discussion  it  is  clear  why  the  statistical  pattern  recognition  is  still  the 
favoured  approach  in  remote  sensing  applications. 

4.2  Statistical  pattern  recognition  -  which  technique  to  use  ? 

In  the  image  classification  context  two  terms  are  commordy  encountered:  feature  class  and 
information  class.  Feature  classes  refer  to  data  clusters  having  similar  statistical  properties 
and  they  may  or  may  not  have  any  bearing  on  objects  that  we  recognise.  On  the  other  hand 
information  classes  refer  to  classes  of  objects  that  we  identify  as  types  of  land  cover. 

The  discrimination  processes  used  for  extracting  feature  classes  are  commonly  called 
'clustering'.  Clustering  is  mainly  used  for  studying  the  data  structures  in  terms  of  separable 
groups  of  data  points.  As  this  process  requires  no  supervision  whilst  training  the  process  to 
identify  objects  it  is  called  'unsupervised  classification'.  This  process  is  not  generally  used  in 
remote  sensing  investigations  as  the  clusters  are  hard  to  relate  to  the  terrain  objects  that  the 
user  intends  to  investigate. 

Information  classes  are  extracted  from  the  images  by  training  a  statistical  process  to  recognise 
objects  from  their  spectral  properties.  The  processes  are  trained  to  discriminate  between 
information  classes  by  providing  a  sufficiently  large  number  of  samples,  usually  more  than 
100,  of  each  information  class.  As  the  user  supervises  the  process  during  training,  the 
technique  is  called  'supervised  classification'.  In  this  report  only  supervised  classification 
techniques  are  used. 

4.3  Digital  Pattern  recognition  versus  aerial  photo>interpretation 

Digital  classification  of  satellite  images  can  produce  a  greater  variation  within  information 
classes  than  visual  interpretation  of  aerial  photographs  or  ground  truthing  can  produce, 
particularly  regions  of  uniform  texture  (Sali  and  Wolfson,  1991,  Hyland  et  al,  1988).  This 
issue  is  explained  in  Klemas  et  al  (1975)  with  examples.  The  finer  resolving  power  of  the 
classifiers  can  often  be  disadvantageous  as  it  causes  graininess  or  'salt  and  pepper  effect'  in 
classification  results. 

To  overcome  the  graininess  of  the  classification  a  post<lassification,  line  preserving 
smoothing  filter  was  passed  over  many  of  the  classified  images  in  this  report.  The  filter  is  a 
part  of  the  ISI-2  image  processing  software.  The  effect  of  this  is  to  remove  any  single  or 
isolated  small  groups  of  pixels  that  represent  the  finer  classification  not  perceived  by  the 
human  eye.  It  does  however  preserve  groups  of  pixels  of  a  prescribed  length  or  greater  that 
could  represent  linear  features  in  the  classified  image.  All  GMLC  classified  results  shown  in 
this  report  are  post-processed. 

4.4  Robustness  of  common  classifiers 

In  developing  classification  procedures  one  may  encounter  following  three  possibilities 
(Lachenbruch  and  Goldstein,1979): 

1.  the  probability  distribution  of  data  is  completely  lunown. 
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2.  the  probability  distribution  of  data  is  known  or  assumed  but  the  parameters  are 
unknowiv 

3.  nothing  whatever  is  known  and  a  distribution  is  not  assumed 

In  remote  sensing  applications  the  first  case  is  very  rare.  If  the  distribution  is  completely 
known  the  posterior  probability  may  be  easily  computed  and  the  pixel  can  be  assigrted  to  the 
class  with  the  highest  probability.  The  second  case  is  very  common  and  usually  a 
multivariate  normal  distribution  is  assumed.  The  unknown  parameters  are  obtained  from  the 
maximum  likelihood  estimates  of  training  areas.  Class  membership  is  assigned  according  to 
the  maximum  likelihood  rule. 

In  the  second  case,  nothing  is  assumed  about  the  underlying  multivariate  distribution  and  the 
procedures  are  called  non>parametric:  that  is.  they  are  distribution  free.  In  principle  non- 
parametric  procedures  should  provide  better  results  if  the  samples  are  large,  as  in  remote 
sensing  applications,  and  when  the  distributions  are  not  normal  (Lachenbruch  and 
Goldstein,1979). 

Two  classifiers  are  well  known  in  classification  or  discriminant  analysis  literature;  Fisher's 
linear  discrimiruint  classifier  (LDC)  and  Gaussian  nruuumum  likelihood  classifier  (GMLC). 
Both  of  these  techniques  provide  optimal  results  when  the  data  have  a  Gaussian  or  Normal 
distribution. 

GMLC  is  by  far  the  most  widely  used  classifier  in  remote  sensing  studies.  By  definition,  the 
classifier  assumes  multivariate  normal  distribution  for  data  points.  Although  this 
assumption  is  violated  by  most  of  the  data  set  we  encounter,  the  technique  works  very  well. 
The  technique  is  fairly  robust  to  violation  of  Gaussian  model  (Richards, 1986). 

LDC  is  twt  commonly  used  in  remote  sensing  investigatiorrs.  However  it  has  certain  qualities, 
apart  from  the  simplicity  of  its  implementation,  which  merit  brief  consideration  here. 
Shettigara  (1991a)  has  compared  the  performance  of  LDC  and  GMLC.  His  study  showed  that 
LDC  is  more  robust  than  GMLC.  LDC  is  widely  and  quite  erroneously  associated  with 
multivariate  distribution.  It  is  true  that  LDC  gives  optimal  results  for  multivariate  normal 
data.  However,  it  is  not  restricted  to  any  particular  distribution.  The  technique  maximises 
distaiKe,  in  spectral  data  space,  between  class  means  (Anderson,1982).  LDC  is  a  legitimate 
non-parametric  technique. 

Anodrer  family  of  non-parametric  techniques  that  have  become  popular  recently  is  based  on 
Artificial  Neural  Networks  (ANNs).  A  partkulitr  kind  of  ANN,  the  multi-layer  perceptron 
(MLP)  can  be  used  for  supervised  classification.  The  MLP  does  not  use  any  data  distribution 
model  and  in  pritKiple  should  be  able  to  fit  the  best  possible  decision  surface  between  two 
classes,  giving  100%  accuracy  for  separable  classes.  In  practice  such  accuracy  is  not  achieved 
for  a  variety  of  reasons;  e.g.  training  is  a  long  process  and  is  often  curtailed.  Some  researchers 
claim  that  ANNs  perform  better  if  they  are  trained  on  properly  pre-processed  data  sets 
(lisaka  and  Russell,1991).  However,  most  cases  documented  in  the  literature  do  not  share  this 
view.  In  goieral,  accuracy  has  been  fourul  to  be  comparable  with  standard  GMLC  techniques 
(Benediktsson  and  Swain,  1990;  Hepner  et  al.,  1990;  Sheldtm,  1990;  Kanellopoulous  et  ai, 
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1991;  Heennann  and  Khazenie,  1992).  The  results  of  using  ANNs  for  classification  are 
presented  later  in  the  sectiort. 

4.5  Choice  of  classes  and  training  areas 

To  classify  an  image  using  supervised  classification,  areas  need  to  be  selected  that  are 
representative  of  classes  chosen.  These  areas,  known  as  traiiung  areas,  are  the  key  to  a 
successful  classification.  When  selecting  training  areas  for  supervised  classification  three 
important  factors  should  be  considered  (loka  and  Koda,1986).  The  training  areas  should 

(a)  fully  reflect  categorised  theme  information  which  is  visually  interpreted  and 
recognised, 

(b)  contain  all  possible  multispectral  features  inherent  in  each  category,  and 

(c)  satisfy  the  a  priori  statistical  assumptions/conditions  of  the  classification 
procedures. 

As  stated,  training  data  have  to  represent  as  much  spectral  variation  within  the  class  as 
possible.  This  is  particularly  highlighted  by  Wright  and  Harris  (1988).  They  found  that  the 
accuracy  of  classification  improved  when  more  than  one  training  area  represented  a  class.  In 
this  study,  initially  51  known  training  areas  were  chosen  using  the  kiwwledge  gained  from 
analysing  aerial  photographs  and  extensive  ground  truthing  of  the  study  area.  All  the  51 
training  areas  are  identified  in  a  theme  file  by  giving  a  uruque  number  to  pixels  in  each 
training  area  (Table  4.1).  All  the  training  areas  represent  a  recognisable  land  cover  (class) 
and  care  was  taken  not  to  overlap  boundaries  into  other  classes. 

The  51  training  areas  were  then  reduced  to  11  super-training  areas  (classes)  initially,  by 
merging  training  sets  as  shown  in  Table  4.1.  Later,  one  of  the  classes,  dull  metal,  was  further 
divided  into  two  classes,  namely  dull  metal  and  specular  metal.  The  reason  for  this  will  be 
explained  later  in  this  section.  The  merging  of  trairung  areas  was  necessary  because  many 
training  areas  represented  a  single  meaningful  physical  class. 

To  decide  how  to  merge  smaller  training  sets  into  super-training  sets,  graphical  and 
statistical  techniques  were  used  apart  from  considering  the  physical  appearance.  The 
multispectral  data  for  each  training  area  were  plotted  as  scattergraiiu  aiul  their  distributions 
relative  to  other  classes  were  used  as  a  guide  to  pool  some  of  the  training  areas.  The  Jeffries- 
Matusita  0-M)  distance  measure  was  also  used  as  a  guide  to  merge  different  training  areas. 
J-M  distance  is  a  statistical  measure  for  determining  the  separability  of  2  classes.  As  the 
separability  increases,  the  J-M  distance  asymptotically  approaches  a  value  of  2.0.  The 
distance  measure  has  the  advantage  of  varying  the  measure  scale  by  stretching  the  smaller 
class  separations  and  contracting  larger  class  separatioiu. 
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Table  4.1  Suminaiy  of  all  the  initial  51  training  areaa 


Qass  name  description 

Theme 

No  of  pixels 

Sugar  Gums 

Aerodynamics  area 

11 

104 

Aerodynamics  area 

12 

126 

Aerodynamics  area 

13 

96 

Boundary  Aerodynamics  aitd  RAAF 

14 

120 

East  contactors  area 

15 

129 

Aleppo  Pines 

NW  of  army  logistics  base 

21 

84 

NW  of  army  logistics  base 

22 

84 

Between  DSTO  and  RAAF 

23 

54 

along  disused  railway  line 

2  km  south  of  above  site 

24 

47 

Open  fields  and  Grassland 

250  m  NW  of  Angle  Vale  Rd 

30 

1374 

and  rail  crossing 

150  m  E  of  Angle  Vale  Rd 

?1 

1692 

and  rail  crossing 

Between  dirt  aitd  bitumen  runway 

32 

240 

1000  m  south  of  old  train  circle 

33 

143 

500  m  east  of  rtear  civil  construction 

34 

266 

100  m  west  of  Elizabeth  South 

35 

395 

train  station 

North  field  of  DSTO  area 

36 

987 

200  m  south  of  runway 

37 

967 

Between  Labs  and  Contractors  area 

38 

535 

Top  NW  comer  of  the  image 

39 

520 

Lawns 

RAAF  oval  -  north 

41 

108 

RAAF  oval  -  south 

42 

96 

DSTO  north  golf  oval 

43 

71 

Elizabeth  Oval 

44 

156 

Dull  metallic  surface 

Metallic  hanger-RAAF  base 

52 

21 

87  labs 

53 

15 

MaintenaiKe  and  workshops 

54 

37 
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Bright  Metallic  Surface 

stores  north  building 

61 

83 

Asbestos  Surface 

77  Labs 

71 

36 

40  Labs  (library) 

72 

16 

102  Contactors 

73 

39 

275  Aerodynamics  area 

74 

16 

71  Labs 

75 

32 

Dark  Apron  -  Ediitburgh  airbase 

west  compass  swing 

81 

40 

south  compass  swing 

82 

54 

east  compass  swing 

83 

36 

visiting  aircraft  apron 

84 

63 

Light  Apron  -  Edinburgh  airbase 

North  of  hangers 

91 

90 

North  of  hangers 

92 

21 

Between  hangers 

93 

56 

North  of  hangers 

94 

33 

South  end  of  runway 

95 

185 

North  end  of  runway 

96 

237 

Specular  Metallic  Surface 

GMH 

101 

591 

dull  hanger  -  Edinburgh  airbase 

102 

20 

Concrete  Surface  •  Edinburgh  airbase 

compass  swing 

111 

38 

engiiw  test  bay 

112 

47 

North  piano  keys 

113 

28 

Barren  Soil  Surface 

east  of  horse  track  and  500  m 

west  of  runway 

121 

57 

south  of  horse  track  and  500  m 

west  of  runway 

122 

89 

Road  150  m  south  of  runway 

123 

64 

horse  track  500  m  west  of  runway 

124 

68 
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Figure  4.1  Mean  spectral  resp<mses  for  11  super  classes  with  the  standard  deviations 

mailied 

4.6  Classifications  using  Maximum  Likelihood 
4.6.1  Qassification  accuracy 

The  confusion  matrix  in  Table  4.2  shows  the  initial  classification  results  from  using 
GMLC.  The  overall  accuracy  of  the  classification  in  the  test  areas  is  82%  that  is  not 
entirely  satisfactory.  It  is  felt  dut  an  error  of  about  10%  is  tolerable,  in  a  subjective 
sense.  Most  errors  are  in  three  classes;  open  fields,  dull  metallic  roof  and  dark  bitumen. 
The  misclassifkation  is  particularly  predominant  amongst  artificial  objects.  However, 
very  few  luitural  objects  are  misclassified  as  artificial  objects,  iiKlicating  that  the 
commonly  occurring  artificial  objects  are  quite  distinguishable  from  natural  objects  in 
spectral  domain. 

The  misclassifkation  in  open  fields  is  due  mainly  to  the  inhomogeneity  in  the  class. 
This  can  be  seen  from  the  individual  training  areas  within  the  class,  as  shown  in 
Table  4.1.  The  class  might  have  included,  as  a  part  of  the  open  fields,  some  sparsely 
vegetated  soil  covers  which  are  classified  as  barren  soil  covers.  As  no  aerial 
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photographs  contemporaneous  with  the  SPOT  images  are  available,  it  is  difficult  to 
ascertain  if  the  confused  areas  were  low  in  vegetation. 

The  confusion  matrix  reveals  few  other  interesting  properties.  Amongst  artificial 
objects  most  of  the  misclassification  occurs  between  objects  of  similar  nature  For 
example,  a  significant  amount  ot  pixels  in  dull  metallic  roof  is  classified  as  bright 
metallic  roof.  Similarly  quite  a  big  number  of  dark  bitumen  pixels  is  misclassified  as 
light  bitumen.  This  miKlassification  can  be  generally  tolerated  as  they  represent 
similar  materials  and  their  differentiation  may  not  be  necessary  for  most  of  the 
applications. 

4.6.2  Qassification  blunders 

The  confusion  matrix  does  not  reflect  the  true  cost  of  classification  or  mis-classification. 
In  some  cases,  although  the  percentage  error  in  misclassification  is  small,  from  the 
defence  point  of  view  the  misclassification  can  be  considered  serious.  Such  errors  are 
grouped  as  blunders.  Blunder  detection  should  become  an  important  part  of  any 
classifier. 

For  GMLC,  there  are  a  couple  of  misclassifications  of  serious  nature.  12.8%  of  dark 
bitumen  pixels  are  misclassified  as  bright  metallic  surfaces.  Spectrally  these  two 
classes  are  quite  different  as  shown  in  Figure  4.1.  The  metallic  surfaces  are  highly 
reflective  in  all  the  baiKis  while  the  dark  apron  is  highly  absorptive  in  all  uonds.  The 
I'M  distance  between  the  classes  is  1.99,  which  indicates  that  the  two  classes  are  quite 
separable.  This  suggests  that  the  misclassification  is  not  due  to  the  overlapping  of 
classes.  A  closer  examination  of  the  dark  bitumen  test  area  revealed  that  there  were 
aircraft  parked  on  the  bitumen  that  were  barely  visible  in  the  original  multispectral 
image.  The  features  in  the  test  area  are  in  fact  stationary  aircraft  on  the  bitumen. 
Aircraft  are  metallic  and  so  show  a  similar  spectral  response  to  the  metallic  class.  As 
such,  the  pixels  have  been  correctly  classified  as  metallic  surface  on  the  dark  bitumen 
surface.  This  demonstrates  the  capability  of  a  simple  classifier  to  detect  small  objects 
of  distinct  spectral  signature  in  relation  to  their  background.  The  object  may  not  be 
visible  due  to  their  small  size  in  relation  to  the  pixel  size  and/or  due  to  mixing  of  two  or 
more  signatures. 

Another  serious  misclassification  has  occurred  in  light  bitumen  class.  3%  for  the 
bitumen  area  (Table  4.2),  which  may  be  normally  considered  insignificant,  has  been 
misclassified  as  asbestos.  These  errors  occur  along  the  boundary  between  the  open  field 
and  light  bitumen  classes.  The  reason  for  this  error  is  suspected  to  be  due  to  mixing  of 
open  field  and  light  bitumen  classes.  In  Figure  4.2  spectral  profiles  for  light  bitumen, 
open  fields  and  asbestos  surfaces  are  shown.  A  profile  for  the  mixed  class  with  equal 
parts  of  open  fields  and  light  bitumen  classes  is  presented.  As  can  be  seen  the  spectral 
profiles,  open  fields  and  light  bitumen  classes  are  distinct  particularly  in  the  infrared 
band.  However,  when  these  two  spectral  classes  are  combined  in  various  proportions  it 
is  easy  to  see  that  the  resulting  pixel  will  have  a  profile  similar  to  that  of  asbestos 
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Figure  4.2  An  illustration  of  the  effect  of  mixing  of  two  classes  •  light  bitumen  surface, 
such  as  runways,  and  open  fields,  such  as  crop  and  pastures. 

Another  significant  misclassification  has  occurred  between  dull  metallic  surface  and 
concrete  surface.  A  hig^  percentage  of  dull  metallic  surface  class  has  been  labelled  as 
concrete  surface.  From  Figure  43  it  can  be  seen  that  the  spread  of  the  concrete  surface 
traiiung  areas  is  relatively  small  and  as  such  this  class  splits  the  dull  metallic  class 
into  two  distinct  classes.  This  indicated  a  need  for  the  dull  metallic  surface  theme  class 
to  be  split  into  two  classes. 


UNCLASSIRED 


•  • 


ERL-0684-RR 


UNCLASSIFIED 


graan  rad  infrarad  pal  pa2 

SPOT  BANDS 

Rguie  Detailed  spectral  profile  of  metallic  surface  and  concrete  theme  classes 

The  J-M  distartce  between  dull  metallic  and  concrete  classes  is  only  1.2059.  Using  the 
relationship  between  the  J-M  distance  and  the  probability  of  error  (Swain  and 
Davis,1978)  the  maximum  error  in  classification  is  estimated  to  be  of  the  order  of 
13.6%,  given  equal  prior  probability  for  die  two  classes.  This  indicates  that  the  two 
classes  are  fairly  separable.  However,  the  actual  error  observed  is  very  high.  One  of 
the  possible  reasons  for  this  is  that  the  probability  distribution  function  (pdf)  assumed 
for  one  or  bofii  the  classes  may  be  wrong.  A  look  at  dte  scattergrams  of  the  dull  metallic 
class  revealed  diat  it  is  bimodal,  which  seriously  violates  the  assumption  that  data 
have  normal  distribution. 

As  a  result  of  this  finding  a  furdier  classification  was  performed,  where  the  dull 
metallic  class  was  split  into  two  classes,  dull  metallic  and  specular  metallic  surface. 
Table  4.3  is  the  coidusion  matrix  for  the  new  classification.  Note  that  the  overall 
accuracy  of  the  image  classification  has  improved  from  87.38%  in  the  first 
classification  to  90.49%  in  dte  second  classification  if  the  accuracy  test  is  done  on  the 
training  area.  However,  if  die  accuracy  is  computed  on  a  number  of  test  areas  the  over 
all  accuracy  drops  slighdy,  from  82  %  to  81.2%.  Bulk  of  die  error  is  in  the  open  field 
class.  If  we  consider  only  the  artificial  objects  the  over  all  accuracy  has  improved  from 
81.2  %  to  873  %  in  the  second  classificatim.  This  level  of  accuracy  is  closer  to  the  tzu’get 
accuracy  of  90%  set  in  the  begiiming  of  the  study.  The  open  field  class  probably  needs  to 
be  further  sub-divided  into  two  or  more  sulxlasses  in  order  to  improve  the  over  all 
accuracy.  The  new  classified  image  is  seen  in  Figure  4.4 
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Maximum  Likelihood  classification 
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Figure  4.4  Result  of  maximum  Iikeiihcud  classification  after  sub-dividing  dull  metal 
class.  The  accuracy  figures  are  with  res-'-i-rt  to  test  areas.  The  overall  accuracy  of  artificial 

objects  is  87.3% 
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4.7  Qassifications  using  Multi-Layer  Perceptron 

4.7.1  Some  notes  on  the  application  of  MLPs  to  classification 

The  classification  power  of  the  multilayer  perceptron  has  been  recogrused  for  a  number 
of  decades,  but  only  since  the  work  of  Rummelhart  et  al  (1986)  has  it  been  possible  to 
train  the  MLP  from  example  data.  A  detailed  definition  of  the  MLP  and  a  description 
of  the  method  of  training  for  this  application  is  given  by  Whitbread  (1992).  Since  the 
application  of  MLPs  to  classification  of  remotely  sensed  data  is  relatively  new 
technology,  some  comments  on  its  limitations  and  difficulties  of  implementation  are 
included  here. 

The  MLP  is  attractive  as  a  classifier  because  it  is  essentially  model  free,  and  during 
training  learns  not  only  the  parameters  of  a  class  but  also  the  data's  probability 
distributions.  In  situations  where  there  are  naturally  occurring  non-normal  distributions 
(which  challenge  GMLC)  it  is  possible  to  produce  an  optimal  classifier.  A  particular 
case  of  non-normally  distributed  data  is  auxiliary  data  such  as  digital  elevation  data, 
which  could  improve  classification  accuracy  when  combined  with  remotely  sensed 
data.  'The  MLP  is  also  of  interest  because  of  the  potential  to  realise  it  in  hardware  as  a 
fast  parallel  classifier.  However,  results  presented  in  this  report  are  based  solely  on 
simulation. 

The  MLP  has  not  been  embraced  by  the  bulk  of  users  of  classification  processes  because  of 
some  limitations  in  implementation.  Firstly,  the  MLP  is  slow  to  learn;  whereas  the 
GMLC  measures  its  required  parameters  by  a  single  inspection  of  foe  training  data,  an 
MLP  learns  its  parameters  by  continued  re-inspection  of  the  training  examples. 
CommoiUy  20,000  inspections  can  be  required  to  achieve  1%  accuracy,  and  currently  most 
MLP  classification  is  done  by  simulation  that  multiplies  overheads.  The  second 
problem  is  that  there  is  no  current  theoretically  based  method  for  choosing  foe  size  of 
the  MLP  necessary  for  a  particular  classification  task,  but  it  is  clear  that  size  is 
important.  The  current  approach  is  to  choose  foe  size  using  heuristics  combined  with 
trial  and  error.  Heuristics  for  choosing  the  MLP  size  for  the  task  of  classifying 
remotely  sensed  data  were  developed  by  Whitbread  (1992).  The  third  problem  is 
reliability.  Internally,  the  MLP  is  a  non-linear  system,  which  makes  it  hard  to  predict 
its  behaviour  as  data  quality  becomes  peer.  In  model  based  systems  (such  as  GMLC)  we 
expect  "graceful  degradation"  of  behaviour. 

Despite  these  limitations,  the  MLP  promises  potentially  better  classification  accuracy 
than  conventional  classifiers,  and  as  well  promises  greater  ease  of  use,  since  it  is  not 
necessary  to  choose  training  sets  that  satisfy  the  a  priori  statistical  assumptions  of  foe 
conventional  procedures  (see  paragraph  4.5  (c)  above). 
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Neural  Network  classification 
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overall  classification  accuracy  78.68  % 

Figure  4.5  Results  of  ANN  classification  using  classes  selected  for  GMLC  classifier. 
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4.7.2  Classification  Accuracy 

The  purpose  of  the  results  presented  here  is  to  show  the  utility  of  the  MLP 
classification  method.  The  results  are  based  on  the  same  training  areas  as  the  results 
from  GMLC  presented  above  and  they  do  not  take  advantage  of  the  possibility  of 
relaxing  the  requirement  paragraph  43(c)  in  the  choice  of  traiiung  areas. 

The  MLP  used  for  classifying  the  muitispectral  data  had  three  (active)  layers 
comprising  50,  6  and  12  rwdes  in  the  first,  second  ai\d  third  layers  respectively.  The 
MLP  was  tramed  by  adjusting  its  internal  weights  using  the  back  propagation  procedure 
of  Rumelhart,  et  uJ.  (1986).  (The  learning  rate  was  0.01  and  momentum  0.0.)  Traming 
was  limited  to  20,000  iteratioiu,  representing  approximately  1600  presentations  of 
examples  per  class.  Samples  are  selected  at  random  for  each  training  iteration. 

Table  4.5  Comparison  of  percentage  of  comet  dasaificationa  by  MLP  and  GMLC 


Theme 

MLP 

GMLC 

Suqar  Gums 

73.33 

85.78 

Alepfx)  Pines 

93.42 

88.82 

Open  Fields 

64.84 

69.28 

Lawns 

99.38 

99.38 

Dirt  Roads 

98.47 

98.47 

Dull  MetaKc  Roof 

50.00 

77.08 

Specular  Metallic  roof 

97.39 

87.58 

Bright  Metallic  Roof 

90.00 

96.67 

Asbestos  Roof 

83.33 

97.44 

Dark  Bitumen 

75.23 

100.00 

Light  Bitumen 

74.16 

78.52 

Concrete 

52.17 

84.78 

AVERAGE 

79.31 

88.65 

Table  4.4  shows  the  cortfusion  matrix  for  MLP  classification  using  the  same  traming 
areas  as  previous  experiments  artd.  Table  45  shows  the  relative  accuracy  of  MLP  and 
GMLC  using  the  same  training  areas.  The  MLP  has  not  been  able  to  better  the  accuracy 
of  GMLC  in  Oiis  test.  Confusion  is  of  the  same  kind  as  GMLC  with  Gums  being 
misclassiried  as  Pines,  barren  soil  classified  as  open  field  and  light  bitumen  being 
classified  as  dark  bitumen.  Other  more  troubling  confusions  occur  between  concrete  arKi 
metallic  reflectors.  This  is  caused  mostly  by  a  very  strong  reflection  from  these 
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surfaces.  Asbestos  is  also  a  problem,  with  some  open  fields  being  confused  with 
Asbestos  rooves.  A  major  contributor  to  the  good  average  performance  in  the  CMLC  case 
is  the  100%  accuracy  for  dark  bitumen,  for  which  the  MLP  only  gets  75%  correct. 

Because  the  MLP  can  handle  an  arbitrary  number  of  inputs,  it  is  possible  to  reconfigure 
it  to  use  texture  in  the  training  areas.  In  this  demortstration  of  classification  that  was 
impractical  because  the  training  areas  were  selected  for  GMLC  and  as  such  were  chosen 
not  Id  have  texture  where  possible,  and  many  training  areas  were  too  snnall  to  make  an 
estimate  of  texture.  The  potential  for  the  use  of  the  MLP  in  this  way  is  discussed 
further  in  the  referatce  by  Whitbread  (1992). 

4.8  Conclusions 

This  section  has  described  a  number  of  pattern  recognition  aids  for  interpreting  satellite 
images  and  has  shown  how  classification  can  be  used  to  extract  man-made  objects.  The 
standard  GMLC  has  performed  better  than  MLP  in  this  study. 
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5  OBJECT  HEIGHT  DETERMINATION 


In  the  last  two  decades  enormous  gains  have  been  made  in  processing  satellite  images. 
However,  the  aiud3rses  of  the  images  have  remamed  mostly  qualitative  in  nature,  compared 
to  the  field  of  photogrammetry.  This  chapter  presents  an  application  of  remote  sensing  for 
height  determination  using  SPOT  images. 

Photogrammetrists  have  been  able  to  extract  the  heights  of  objects  from  aerial  photographs 
using  parallax  in  stereo-pair  photographs.  The  lengths  of  the  shadows  cast  by  the  objects  are 
also  used  to  determine  the  heights.  If  the  sun  and  sensor  geometry  is  known  it  is  fairly  simple 
to  establish  a  relatiotuhip  between  the  shadow  width  and  the  height  of  the  objects.  The 
above  usages,  however,  are  confined  to  high  resolution  photographs  or  images. 

5.1  Shadow  •  an  important  part  of  images 

Shadows  form  a  unique  part  of  any  image.  They  are  easily  detectable  as  they  show  a  low 
intensity  in  all  the  multispectral  bands.  Shadows  play  a  dominant  role  in  image  statistics 
which  affect  image  enharKement  and  pattern  recognition.  Shadows  contain  3-D  information 
of  objects  and  they  have  been  used  in  object  identification,  terrain  classification  and 
geological  mapping  (Curran,1985). 

Detection  of  objects  by  their  shadow  structures  has  been  performed  by  many  workers  in  the 
photogrammetric  community  (Venkateswar  and  Chellappa,1990;  Irvin  and  McKeown,1989; 
Huertas  and  Nevatia,1988).  All  have  utilised  the  nature  of  shadows  around  buildings  to 
interpret  the  structure  of  the  building.  An  edge  detection  routine  has  been  used  by  Huertas  and 
Nevatia  (1988)  that  looks  for  different  types  of  shadows  that  represent  the  comers  of 
buildings  in  the  aerial  photographs.  One  of  the  problems  Huertas  and  Nevatia  (1988) 
erKountered  in  their  study  was  in  identifying  the  shadows.  Their  technique  failed  to  identify 
some  areas  of  shadow  or  included  areas  that  were  ttot  in  fact  shadow.  Venkateswar  and 
Chellappa(1990)  have  used  knowledge  based  procedures  to  overcome  some  of  these  problems. 

Although  photogrammetrists  have  successfully  used  shadows  in  aerial  photographs  for 
height  determination  and  object  identification,  the  use  of  satellite  images  for  similar 
purposes  is  not  seen  in  the  literature.  The  main  reason  is  that  the  resolution  of  the  civilian 
satellite  images  is  much  coarser  than  die  aerial  photographs  and  the  shadows  are  not  fully 
defined  for  short  and  commonly  occurring  objects. 

5.2  Sun  •  satellite  geometiy  and  the  equations 

Figure  5.1  shows  the  sun-satellite  geometiy  as  an  end  view  (2-D  display  of  3-D  geometiy  for 
simplicity)  for  the  two  SPOT  images  used  in  dtis  study.  As  illustrated  in  the  figure  the 
shadow  part  seen  by  the  sensor  is  different  for  the  two  images.  The  width  of  the  shadow, 
measured  along  the  normal  to  the  object,  depends  on  the  azimuths  of  the  sun,  image  scan  line 
and  the  object.  The  objects  are  rows  of  trees  in  this  study  (Figure  5.2). 
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In  principle  the  procedure  that  is  presented  in  this  section  is  suitable  for  any  object  that  casts 
detectable  shadows  of  sufficient  length.  In  the  study  area  rows  of  trees  cast  shadows  that 
could  be  used.  Buildings  were  either  not  tall  enough  or  long  enough  to  cast  a  consistent  shadow. 
For  this  reason  the  shadow  detection  and  height  analysis  were  initially  performed  on  rows  of 
trees.  After  establishing  the  relationship  between  the  shadow  widths  and  the  measured 
heights,  the  heights  of  the  unknown  objects  (buildings)  were  determined. 

For  relating  sha  i  rv  width  to  the  object  (tree)  height  a  few  basic  assumptions  were  made. 
Firstly,  the  object  is  assumed  to  be  vertical,  that  is  the  object  is  perpendicular  to  the  earth's 
surface.  Secondly,  the  shadows  are  assumed  to  be  cast  from  the  top  of  the  object  and  that  the 
top  of  the  object  is  also  the  last  part  of  the  object  seen  from  the  sensor  before  the  shadow 
begins.  This  assumption  is  not  entirely  satisfactory  because  there  could  be  parts  of  object, 
between  the  illuminated  part  of  the  object  and  the  ground  shadow,  not  illuminated  by  the  sun 
but  seen  by  the  sensor  due  to  the  scattered  light.  The  computation  assumes  that  the  parts  of 
the  object  not  directly  illuminated  by  the  sun  as  part  of  Bte  shadow.  Thirdly,  it  is  also 
assumed  that  the  shadow  starts  from  the  tree-trunk  lir\e  on  the  grourtd.  Fourthly,  as  shown  in 
Figure  5.1,  it  is  assumed  that  if  the  sensor  and  the  sun  are  on  opposite  sides  of  the  object,  then 
the  sensor  is  able  to  see  the  entire  shadow.  Lastly,  the  surface  that  the  shadow  fails  on  is 
even  and  level. 
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Rgure  5.1  End  view  of  the  sun-satellite  configuration  as  seen  during  inuging 


Image  2  SPOT  1 
band3;  panchromatic  25-4-87 


SPOT  1  1 

21-4-87  bandl:  muItispectral-IR 
band2:  panchromatic 
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For  the  panchromatic  and  multispectral  image  of  21st  April  1987  the  data  are  as  follows: 


0sa  =  23.4°  and  080  =  555° 

#sw  =  13.2°  and  ♦san  =765° .  (5.8) 

Using  equations  (5.1)  to  (5.4)  and  (5.8)  we  get 

ht  =  0.76  s  (5.9) 


If  the  tree  or  shadow  line  is  at  an  angle  to  the  image  scan  tine,  an  additional  correction  needs 
to  be  applied  to  equation  5.9.  This  is  because  of  the  pixelisation  of  shadow  zones.  The  issue  is 
explained  in  Appendix  II.  The  shadow  width  in  equation  5.9  needs  to  be  divided  by 
cos(^can)<  where  0scan  the  angle  between  the  scan  lines  and  the  tree  line.  For  the  rows  of 
trees  considered  ^an  =  ^5°.  The  relation  used  in  this  study  is: 

hr  =  0.76  s/cos(6scan)  =  0.79  s 

=  ks  .  (5.10) 

where  k  is  a  constant  for  obiects  of  fixed  azimuth  in  an  image. 

5.3  Relationship  between  shadow  width,  pixel  size  and  intensities 

The  above  relationships  assume  that  the  shadow  zones  are  sharply  defined  and  the  pixel 
size  is  iitfinitesimally  small.  However,  in  practice  neither  the  shadow  zones  are  well  defined 
nor  the  pixel  sizes  are  negligibly  small  in  relation  to  shadow  widths.  The  following 
discussion  presents  simple  relationship  between  pixel  intensities  and  shadow  boundaries. 

If  the  brightness  of  the  shadow  (object)  is  Iq  and  that  of  the  background  is  1),,  then  the 
brightness  of  a  pixel  may  be  approximated  to: 

If  =  fl0  +  (l-01b  .  (5.11) 

where  p  is  the  area  fraction  of  shadow  in  the  pixel. 

If  Iq  and  Ib  are  known  from  pure  shadow  and  background  pixels,  then 

f  =  (If-Ib)/(l0-Ib)  .  (5.12) 

There  will  normally  be  a  threshold  fmin  such  that  if  &f|n:n  die  pixel  is  classed  as 
shadow.  If  not  it  is  classed  as  background.  Qearly,  the  representation  of  sh.idow  in  the  image 
depends  on  the  threshold  chosen. 

5.3.1  The  role  of  pixel  dimension 

As  we  are  considering  the  shadows  cast  by  linear  objects  of  length  much  greater  dian 
that  of  the  pixel  dimension,  we  will  not  lose  any  generality  if  we  consider  the 
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relationships  only  along  a  profile,  say  X  axis,  orthogonal  to  the  object.  If  the  shadow  of 
length  s  of  uniform  intensity  is  centred  at  the  origin  of  the  profile  and  the  pixel  size  p  is 
smaller  than  the  shadow  size  the  intensity  of  pixels  along  the  profile  for  various  x  is 
given  by 


f=« 


(x*{s+p)/2) 

P 


(-x-»(s-t-p)/2) 
P 


fori!:Ei<x^ 
2  2 

for±£i<xS<2:2i 
2  2 

for^cx^ 

2  2 

forx^lilEi 

2 


(pixeioutsideobject) 

(pixel  partially  on  the  object) 
(pixel  wholly  within  theobject) 
(pixel  partially  on  theobject) 
(pixel  outside  object) 


(5.13) 


If  the  threshold  is  fmin  Ihen  the  estimated  width  w  of  the  shadow  of  width  s  will  be 


w(p,fmin)  S'*'P(l'2pfmin)  .  (514) 

or  in  terms  of  estimated  tree  heights  he$t  actual  tree  heights  ht,  using 
relation  equation  5.10 

hest  =  ht  +  kp(l-2fmin)  .  (515) 

which  is  a  linear  relationship  between  h^gt  ht  with  an  intercept 
kp(l-2finin)  ^  a  slope  1.  If  fmin  is  very  small,  the  intercept  is  rou^y  kp  and  if  fmin 
is  very  high  (near  one),  the  intercept  is  roughly  -kp.  The  intercept  is  zero  for  fmin  =  i/2, 
which  means  that  the  boundary  between  the  shadow  and  the  background  is  half  way 
between  their  respective  intensities.  In  that  case  the  estimated  height  is  equal  to  the 
actual  height.  This  relation  assumes  that  p<s,  object  and  background  intensities  are 
uniform,  and  that  the  measurement  errors  are  normally  distributed.  In  actual  practice 
the  latter  two  assumptions  may  not  be  true.  As  sudi,  the  above  equations  may  only  be 
used  as  a  guide  to  arrive  at  an  optimal  fmin'  Oirough  experiments,  such  that  the 
intercept  and  the  slope  of  the  regression  line  in  eqtiation  5.15  are  close  one  and  zero 
respectively. 

5.4  Shadow  Segmentation 

5.4.1  Choosing  the  threshold  for  delineating  shadow  zones 

In  high  resolution  images  one  might  find  a  separate  peak  in  the  histogram 
corresponding  to  shadow  areas,  as  described  by  Otsu  (1979).  In  such  cases  it  is  easy  to 
separate  the  shadow  zones.  In  lower  resolutitxi  images  it  is  not  common  to  find  a 
separate  peak  for  the  shadow  areas.  The  current  study  required  a  general  procedure  for 
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delineating  shadows  falling  on  various  types  of  backgrounds.  The  backgrounds  included 
lush  grass,  barren  soil,  white  concrete  and  black  asphalt  covers.  Experiments  were 
conducted  by  choosing  various  thresholds  by  varying  a  in  the  equation 

^min  =  |t-ao  .  (5.16) 

where  p  and  a  are  mean  and  standard  deviation  of  the  image.  Note  that  as  a  increases 
the  threshold  decreases.  The  weight  a  was  varied  from  1.6  to  0.8  in  steps  of  0.1.  These 
two  limits  were  found  adequate  as  at  a  threshold  of  1.6o  most  of  the  shadow  areas  were 
mis<lassified  as  non-shadow  areas  and  when  a  threshold  of  0.8  a  is  used  must  of  the 
non-shadow  areas  were  classified  as  shadow  areas. 

5.4.2  Measurement  of  shadow  width 

From  the  accuracy  point  of  view  it  would  be  better  if  the  shadow  width  could  be 
determined  from  SPOT  panchromatic  baruls  which  have  a  resolution  of  10  m.  However, 
in  panchromatic  bands  both  the  trees  and  shadows  appear  equally  dark  and  the 
separation  of  the  trees  and  shadows  is  not  possible.  However,  in  the  infrared  batvl  of 
the  multispectral  image  (Figure  5.3),  which  has  a  resolution  of  20  m.,  the  trees  show  a 
significantly  higher  reflectance  than  the  surroundings  and  the  shadow  zones.  In 
principle  only  the  infrared  band  would  suffice  to  determine  the  shadow  width,  but 
with  a  lesser  precision  due  to  its  coarser  resolution.  In  order  to  improve  the  accuracy  of 
the  estimate  of  the  shadow  width  it  was  decided  to  use  the  panchromatic  and  infrared 
bands  together.  Another  reason  for  using  the  combiiuition  is  for  shadow  detection 
accuracy.  A  low  value  in  infrared  band  does  rtot  necessarily  mean  that  it  is  a  shadow 
zone.  For  example  bitunwn  surface  gives  a  low  reading  in  infrared,  but  a  relatively 
higher  value  in  panchromatic  images.  A  combination  of  panchromatic  and  irtirared 
baruls  would  detect  shadow  zones  better. 

The  image  data  consisted  of  an  ittfrared  barul  of  a  SPOT  multispectral  image  and  two 
panchromatic  bands  taken  on  two  different  days,  as  shown  in  Figure  5.1.  The  three 
bands  were  co-registered  to  form  a  3-barKl  image  using  control  points  at  the  grourtd  level. 
Initially,  for  a  chosen  threshold,  fmiiv  shadow  areas  that  were  coiiunon  to  all  the  three 
bands  were  delineated.  Using  the  same  fnun  shadow  areas  conrunon  to  only  the  infrared 
band  and  the  parKhromatic  band  of  image  1  (Figure  5.1)  were  determined.  No 
significant  difference  was  found  between  the  shadow  zones  determined  in  the  two 
experiments.  For  this  reason  only  the  two  bands  of  image  1  (Figure  5.1)  were  used  for 
further  investigation. 

Figure  5.4  shows  an  example  of  the  shadow  segments  delineated  from  the  images.  Using 
the  segmented  shadow  areas  an  average  shadow  width  is  measured  for  each  row  of 
trees.  As  can  be  seen  from  Figure  5.4,  the  edges  of  the  shadow  areas  are  not  straight. 
This  is  because  of  the  alignment  of  die  rows  of  trees  at  an  angle  to  the  scan  lines  arul 
also  due  to  the  coarse  pixel  resolution.  Due  to  the  jagged  nature  of  the  shadow 
boundaries  the  estimation  of  the  width  is  complicated.  The  average  width  was 
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estimated  by  measuring  the  area  covered  by  each  shadow  zone  and  then  dividing  it  by 
the  length  of  the  zone. 


In  order  to  accomplish  the  above  procedures  the  shadow  zones  were  vectorised  using  the  4r , 

program  RASVEC®  available  on  the  Intergraph  workstation.  The  areas  bound  by  dw 

polygons  corresponding  to  the  shadow  areas  were  calculated  using  the  MicroStation 

CAD  software.  Shadows  of  length  less  than  50  m  were  neglected.  From  the  shadow  ^ 

width  estimates  of  the  tree  heights  were  computed  using  equation  5.10. 
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Figure  5.3  Infrared  SPOT  image  of  the  study  area.  The  rows  of  trees  used  in  the  study  are 
marked  and  the  buildings  whose  heights  are  estimated  using  the  tree  shadows  are  shown  at 

the  lower  right  hand  comer. 
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5.5  Result  and  discussion 

Most  of  the  shadow  zones  shown  in  Figure  5.4  are  correctly  identifiable  as  cast  by  the  rows  of 
trees.  However,  false  identification  of  shadow  areas  has  also  occurred.  The  dark  bitumen  at 
the  west  of  the  image  and  some  wet  market  garden  areas  in  the  south  western  part  of  the 
image  (Figure  5.3)  were  mis-classified  as  shadows.  In  this  study  no  automatic  procedures 
have  been  developed  to  isolate  'false  shadow'  zorws. 


Figure  54  Shadow  zones,  shown  as  polygons,  delineated  using  the  thresholding 
procedure  on  the  infrared  and  the  panchronutic  bands 

There  are  some  rows  of  trees  that  are  parallel  to  the  principal  plane  of  the  sun  or  the  sun 
azimudi  which  do  not  cast  shadow.  No  height  estimation  could  be  made  for  rows  of  trees 
with  such  an  orientation. 

Eight  sites  were  selected  for  tree  height  estimation.  At  these  eight  sites  tree  heights  were 
also  measured.  Table  5.1  siunmarises  the  measured  tree  heights  aiul  the  estimated  heights 
from  the  shadows. 
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Tabu  5.1  Estinuted  and  actual  tiM  haighls 


site 

no. 

Shadow  length  m 

Estimated  tree 
h«ght  m 

Measured  average 
tree  hei^t  m 

Standard  deviation 

of  measurement  m 

2 

29.720 

21.82 

17.90 

1.41 

1 

27.635 

20.28 

17.45 

234 

3 

35.390 

25.98 

15.00 

137 

4 

29.705 

21.81 

17.% 

1.35 

5 

25.191 

18.50 

15.82 

1.11 

7 

11.119 

8.25 

11.76 

0.64 

6 

28323 

20.80 

16.14 

1.26 

8 

25.359 

21.55 

17.11 

0.63 

Figures  5.5  shows  the  regression  lines  of  estimated  heights  computed  horn  shadow  widths 
determined  by  using  different  thresholds  and  actual  tree  heights  are  shown.  For  the  sake  of 
clarity,  regression  lines  and  data  points  for  only  a  few  thresholds  are  plotted. 

Notice  that  the  correlation  between  the  observi^d  heights  and  the  estimated  heights 
deteriorate  drastically  when  the  threshold  is  increased  from  l.2o  to  l.lo.  The  variation  of 
height  estimates  are  shown  for  all  the  sites  with  these  two  thresholds.  The  height  estimates 
increase  systematically,  as  expected,  for  all  the  sites  except  site  5.  The  increase  in  height 
estimate  for  site  5  is  dis-proportionately  high.  This  is  due  to  the  iiKlusion  of  non-shadow 
areas  in  the  shadow  estimates.  Notice  that  the  regression  line  for  any  of  the  cases  involving 
thresholds  up  to  l.lo  do  not  pass  through  the  origin  as  ideally  we  would  have  liked  to.  To 
achieve  an  intercept  of  zero  the  threshold  has  to  be  varied  between  0.9o  arul  0.8o.  However, 
as  we  increase  the  threshold  above  1.2o  background  areas  are  increasingly  classed  as  shadow 
areas.  Thus  the  increase  of  threshold  beyond  12o  is  not  justifiable. 

The  coefficient  of  determination  (r^)  for  thresholds  higher  than  1.2a  is  high  (Figure  53). 
This  iiKlicates  that  foe  shadow  estimates  are  very  well  correlated  (correlation  coefficient  of 
0.832  for  threshold  1.2o)  to  the  measured  tree  heights.  One  point  worth  considering  is  that 
whether  the  number  of  points  used  for  the  regression  is  large  owu^.  The  probability  of 
getting  a  correlation  of  above  03  is  less  than  6%  when  foe  two  variables  are  urKorrelated,  and 
foe  number  of  measurements  involved  is  more  that  6  (Taylor,  1982).  This  suggests  foe  results  of 
regression  are  reliable. 
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Hguic  5^  RcgreHion  lines  for  different  thresholds.  Notice  that  the  estinuted 
heights  vary  systematically  for  all  the  sites  except  for  site  5.  Th  drastic  variation 
for  site  5  is  attributed  to  the  incluritm  of  background  areas  when  the  threshold  is 

increased  from  1.2  to  l.lo. 

Figure  5.6  shows  how  estimated  heights  vary  with  various  thresholds.  Each  line  in  the  plot 
corresponds  to  a  row  of  trees  and  a  location.  The  plot  reveals  that  all  the  estimates  vary 
sharply  between  1.2o  and  l.lo,  indicating  that  non-shadow  areas  are  getting  ixKorporated  in 
the  shadow  estimates.  This  point  is  vividly  illustrated  by  the  site  5  estimate.  As  explained 
above  the  estimate  for  site  5  jumps  drastically  between  1.2  and  l.lo.  This  indicates  that  the 
optimum  threshold  for  separating  shadows  horn  the  background  exists  between  1.2o  to  l.lo. 

It  is  also  interesting  to  note  that  as  the  ttueshoid  is  increased  or  decreased  from  1.2o  the 
height  estimates  tend  to  be  asymptotic.  As  such,  the  estimates  for  rows  of  trees  wifoin  a  range 
of  heights  tend  to  emverge  to  a  common  value.  These  properties  are  currently  being 
investigated  using  theoretical  modeto.  However,  some  heuristic  explanation  for  this  property 
may  be  given  here. 
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In  Figure  5.7  intensity  curves  are  sketched  for  shadow  zones  of  sharp  boundaries  aiul  varying 
widths.  The  curves  mimic  the  smoothing  of  sharp  boundaries  due  to  the  point  spread  functions 
of  the  sensors.  As  can  be  seen,  at  A  and  B  the  intensity  curves  tend  to  overlap  arel  at  O  they 
show  maximum  separation.  Considering  the  noise  inherent  in  images  it  may  be  expected  that 
the  closely  spaced  intensity  curves  can  be  inseparable  at  thresholds  A  and  B.  This  explains 
the  convergeiKe  of  estimated  tree  heists  at  two  extreme  duesholds. 


THRESHOLD  (STANDARD  DEVIATION) 


ACTUAL  TREE 
HEIGHTS 

11  757 

15  817 
15  928 
17  507 
17  717 
17  895 
18.015 


SITE 

7 

5 

6 
1 

8 
2 
4 


Figure  5.6  Variation  of  estimated  tree  height  with  threshold.  Notice  that  the 
estimated  heights  converge  at  two  extreme  thresholds.  Also  the  gradient  variation  is 

maximum  between  1.2  and  Uo 

Figure  5.8  is  a  three  dimensional  (3-D)  display  of  the  tree  heights  interpreted  from  the 
shadows  in  the  image.  The  heights  of  the  rows  of  trees  has  been  vertically  exaggerated  by  a 
factor  of  two.  The  3-D  display  provides  additional  information  apart  from  highlighting  the 
tree  heists.  The  gaps  between  the  rows  of  trees  get  exaggerated,  which  otherwise  may  go  im- 
noticed.  For  example,  the  rows  of  trees  on  the  extreme  left  hand  side  of  the  image  show  a  gap 
which  is  not  evident  in  the  2-D  display  in  Figure  3.2.  The  other  rows  of  trees  also  give  the 
impression  of  having  numerous  gaps.  However,  those  could  be  artefacts  created  by  the 
pixelisation  of  shadows. 
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Figure  3.7  A  sketch  of  intensity  variation  across  shadow  boundaries  to  demonstrate  the 

variation  of  separability  at  various  thresholds.  The  vertical  lines  1,  2  and  3  represent 
the  three  sharp  boundaries  between  shadow  and  background.  The  three  curved  lines 
represent  the  intensity  variation  as  seen  by  the  sensor  across  the  three  boundaries.  The 
lines  A,  B  and  O  represent  the  three  thresholds  and  their  thickness  represents  the 
possible  noise  level.  At  thresholds  A  and  B  the  separation  of  the  three  curved  lines  is 
difficult  to  achieve,  particularly  considering  the  noise.  At  threshold  O  the  curves 
stand  separated. 


Figure  5.8  A  duee  dimensional  display  of  some  rows  of  trees  in  the  DSTO  area. 


UNCLASSIFIED 


47 


•  •  • 


UNCLASSIFIED 


ERL-0694-RR 


BoidiBcl  BirfMiigl  BidUlnc3 
g  Etthnated  Height  Q  Actual  Height 


Figure  5.9  The  plot  of  estimated  and  actual  building  heights  of  some  industrial  buildings 

Based  on  the  above  analysis  the  regression  line  corre.>ponding  to  1.2o  was  taken  as  the 
optimum  calibration  line  (Figure  55)  to  relate  the  estimated  height  from  equation  5.10  to  the 
actual  height.  Using  the  widths  of  shadows  cast  by  the  industrial  buildings  in  the  south-east 
of  the  image  (Figure  53)  the  apparent  heights  were  estimated  using  relation  (equation  5.10). 
The  actual  heights  were  then  estimated  using  the  calibration  line  of  Figure  5.5.  The  result  is 
shown  in  Figure  5.9.  As  can  be  seen  the  estimated  height  closely  follows  the  actual  heights.  A 
high  sub-pixel  accuracy  of  die  order  of  one  third  of  the  pixel  size  has  been  achieved  in  this 
experiment. 

5.6  Some  limitations  of  the  technique 

The  technique  is  useful  for  estimating  heights  of  extended  objects.  The  technique  can  not  be 
used  if  the  object  does  not  cast  a  shadow,  for  example  when  the  extended  side  of  the  object  is 
parallel  to  the  principal  plane  of  the  sun. 

Area  3  (Figure  53)  was  not  used  in  this  study  as  it  was  found  out  early  in  the  investigation 
diat  die  shadow  width  delineated  by  the  process  was  exceptionally  large  compared  to  the 
height  of  the  trees.  At  site  3  two  rows  of  trees  were  found.  It  is  possible  that  the  row  of  trees 
casting  the  first  shadow  is  it  self  mostly  hidden  by  the  second  row  of  trees.  This  will  suppress 
the  infrared  reflectance  from  the  first  row  of  trees  and  the  two  shadow  zones  will  appear  as 
one  as  cast  by  the  second  row  of  trees.  Due  to  the  coarsoiess  of  the  images  it  is  difficult  to 
ascertain  the  actual  situation.  However,  it  is  important  to  eliminate  such  samples  from  the 
regression  analysis. 

It  must  be  also  realised  that  the  data  used  in  this  study  have  been  of  a  limited  height  range 
due  to  die  rows  of  trees  available  in  the  area.  A  large  range  of  heists  would  have  been  more 
suitable  for  the  study. 
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6  NEW  IMAGE  ENHANCEMENT  TOOLS 

Many  of  the  deciskxis  by  defence  agencies  or  resource  industries  are  done  by  visual  pattern 
recognition  as  oppiMed  to  dw  numerical  techniques  explaiiMd  in  section  5.  Image  enhancement 
plays  a  major  role  in  improvli^  the  visual  pattern  recognition  in  the  images  (Woodruff  and 
Ne%vsam,  1992).  The  background  discriminant  analysis  technique  described  in  section  4  is  a 
toot  in  which  objects  of  interest  are  enhanced  in  a  multispectral  image  by  suppressing  the 
contribution  of  background  information.  In  this  section  2  oth«  issues  in  image  enhancement  are 
addressed. 

6.1  Smooth  image  zooming 

The  commercial  satellite  images  have  coarser  resolution  compared  to  aerial  photographs 
which  are  conventionally  used  in  defence  for  pattern  recognition  purposes.  While  displaying 
such  images  the  objects  look  blocky  making  the  recognition  of  foe  objects  difficult.  The 
problem  becomes  more  severe  as  foe  observer  zooms  into  a  snnall  area.  While  no  extra 
mformation  can  be  created  in  foe  blocky  image  once  it  is  created,  one  can  improve  foe 
presentation  of  foe  image  to  assist  foe  human  observer  in  urulerstanding  foe  image.  Image 
smoothing  while  zooming  (Woodruff  arKl  Newsam,  1992)  is  one  way  to  reduce  foe  blocking.  In 
his  approach,  Newsam  (1992)  used  a  smooth  interpolant  to  sub-sample  an  image  to  higher 
apparent  resolution.  An  approximation  is  then  determined  to  foe  interpolant  using  FFTs. 

Using  a  similar  approach  as  that  of  Newsam  (1992),  in  this  study  cubic  convolution  is  used  for 
interpolation  while  zooming.  Cubic  convolution  is  a  process  originally  developed  for  Landsat 
image  reconstruction.  Cubic  convolution  is  simply  another  interpolation  technique 
implemented  by  convolving  foe  image  samples  with  a  smooth,  spatially  limited  one- 
parameter  functioru  constructed  by  joitung  cubic  polynonuals  in  a  spline  like  fashion,  it  is 
generally  accepted  that  this  interpolation  process  gives  a  good  compromise  between 
complexity  of  computation  and  accuracy  (Billingsley,1983).  The  technique  is  generally 
superior  to  the  standard  image  restoration  procedures  arKl  it  can  be  adapted  to  the  frequency 
content  of  foe  image  (Park  and  Schowengerdt,1983). 

In  foe  field  of  remote  sensing  the  cubic  convolution  interpolation  is  extensively  used  for 
resampling.  The  procedure  is  generally  available  in  commercial  software  for  registering 
images  of  two  different  pixel  resolutions.  Due  to  foe  increased  processing  speed  and  foe 
possibility  of  subsampling  foe  images  in  the  main  memory  in  near  real  time  exists  in  most 
machines. 

An  existing  command  line  procedure  in  ISI-2  image  processing  software  on  an  Intergraph 
workstation  was  adopted  for  smooth  zooming.  The  existing  utility  'image  to  image'  in  ISI-2 
can  be  used  to  subsample  an  image  using  cubic  convolution  procedure.  The  input  parameters  for 
the  utility  irudude  the  displayed  image  name,  foe  name  of  the  output  (smooth  zoomed)  image 
to  be  displayed,  and  a  file  contaiiung  a  table  of  ground  control  points.  In  foe  normal 
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application  of  the  utility  the  ground  control  points  are  selected  manually  by  comparing  the 
image  to  be  resampled  and  the  reference  image.  In  the  current  work  the  table  (rf  ground  control 
points  was  generated  automatically  based  on  the  zoom  required.  Table  6.1  demonstrates  how 
the  table  is  generated.  The  whole  sequence  of  generating  the  ground  control  points  table  and 
smooth  zooming  of  the  image  can  be  implemented  as  a  command  procedure  and  assigned  to  a 
function  key.  Depending  on  the  size  of  the  image  aitd  the  processor  speed  the  display  can  be  in 
near  real  time. 


Tabic  6.1  Ground  control  points  for  zooming  by  a  factor  X 


TT 


♦) 


m 


Ar 


GCP  location 

Input  Image 
line 

Input  Image 
pixel 

Zoomed  Image 
line 

Zoomed  Image 
pixel 

top  left  comer 

1 

1 

1 

1 

top  right  comer 

1 

n 

1 

n.X 

bottom  left 

comer 

m 

1 

mX 

1 

bottom  right 
comer 

m 

n 

mX 

nX 

Figure  6.1  is  for  demonstrating  how  an  image  may  be  zoomed  in  to  achieve  a  magnification  of 
8.  Figure  6.1(a)  shows  a  sub-image  of  the  GMH  industrial  complex  adjoining  the  DSTO  area. 
The  image  is  shown  at  its  full  resolution  of  10  m.  Figure  6.1  (b)  is  the  same  image  degraded  to 
80  m  resolution,  which  is  equivalent  to  a  compression  factor  of  64.  Each  pixel  in  Figure  6.1  (b) 
is  a  mean  of  a  8x8  window.  Figure  6.1  (c)  is  the  smooth  zoomed  image  derived  from 
Figure  6.1(b)  using  cubic  convolution.  In  Figure  6.1  (d)  smooth  zooming  is  achieved  by  the 
procedure  of  Newsam  (1992).  As  can  be  seen  the  reconstructed  Figures  6.1  (c)  and  (d)  closely 
resen\ble  the  origirud  full  resolution  image  of  10  m.  resolution  in  Figure  6.1  (a). 

At  this  stage  the  results  of  smooth  zooming  from  the  two  techniques  have  not  been  compared 
in  any  quantitative  sense  The  purpose  of  this  example  is  simply  to  demonstrate  the  use  of  an 
existing  remote  sensing  tool  for  a  fairly  complicated  mathematical  operation.  The  results 
from  the  two  techniques  are  comparable. 

Figure  6.2  is  similar  to  Figure  6.1.  The  magnification  in  this  case  is  16  which  is  equivalent  to 
256  fold  data  compression  Clearly,  the  smooth  zooming  has  not  restored  the  image  in 
Figure  6.2  (b)  to  the  quality  of  Figure  6.2  (a).  The  zoomed  images  in  Figures  6.2(c)  and  (d)  also 
suffer  form  the  distracting  contouring  effect.  The  contouring  effect  can  be  reduced  to  some 
extent,  if  desired,  by  dithering  0ain,1989). 

From  dte  above  two  examples  it  may  be  recommarded  that  up  to  8  tinnes  magnification  can  be 
achieved  using  smooth  zooming  without  signihcantly  degrading  the  image.  Near  real  time 
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smooth  zooming,  using  cubic  convolution,  can  be  implemented  on  most  remote  sensing  image 
processing  systems  using  vendor  provided  utilities. 


(a)  (b) 


(c)  (d) 

Figure  6.1  Smooth  zooming  by  a  factor  of  4. 

(a)  the  original  image  of  10  m  resolution 

(b)  degraded  image  with  40  m  resolution 

(c)  image  in  (b)  is  smooth  zoomed  by  a  factor  4  using  cubic  convolution, 

(d)  image  in  (b)  smooth  zoomed  by  a  factor  4  using  the  interpolation 
technique  of  Newsam(1992) 
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(c)  (d) 

Figure  6^  Smooth  zooming  by  a  factw  of  S. 

( a )  the  original  image  of  10  m  resolution 

(b)  degraded  image  with  80  m  resolution 

(c)  image  in  (b)  is  smooth  zoomed  by  a  factor  8  using  cubic  convolution, 

(d)  image  in  (b)  smooth  zoomed  by  a  factor  8  using  the  interpolation  technique 
of  Newsam(1992) 
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6^  Multispectral  image  enhancement  by  merging  higher  resolution 
panchromatic  data 

The  availability  of  hi^  resolution  data  in  the  form  of  panchromatic  image  data  or  scanned 
aerial  photograph  has  given  rise  to  a  host  of  procedures  to  enhance  coarser  resolution 
multispectral  images.  By  merging,  the  advantage  of  having  both  spectral  and  spatial 
resolution  is  gained.  This  will  enhance  the  target  recognition  task.  Shettigara  (1992)  has 
shown  that  the  merging  process  can  be  achieved  in  one  simple  step  of  linear  transformation. 
Due  to  dnis  simplification  it  is  possible  to  achieve  merging  of  data  in  near  real  time  and 
enhance  the  innage  cm  the  screen  at  the  location  of  interest. 

To  demonstrate  the  power  of  merging  the  multispectral  SPOT  image  of  air  base  (figure  63a) 
was  merged  with  the  panchromatic  image  taken  at  the  same  time  (figure  63.b).  As  can  be 
seen,  merging  has  enhanced  d%e  parked  aircraft  on  the  apron,  which  is  not  clearly  visible  in 
the  original  multispectral  image. 

The  merging  was  achieved  in  3  steps: 

1.  the  total  intensity  component  of  (a)  was  computed, 

2.  the  total  intensity  comporvent  was  substituted  by  dw  panchromatic  data  and, 

3.  the  higher  resolution  information  content  in  die  new  intensity  component  is  re¬ 
distributed  to  the  multispectral  bands.  The  resultant  image  is  shown  in  Figure  63(b). 

Data  merging  can  be  achieved  in  many  other  ways.  For  more  details  the  reader  may  refer  to 
Shettigara  (1992). 
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(a) 


enhanced  using 
SPOT 

w 

panchromatic 

image 

origional  SPOT  multispectral 
image  (20m  pixels) 


enhaiiced  SPOT  multispectral 
image  (10m  pixels  apparent) 


ligure  6.^  Merging  of  panchromatic  image  with  the  SPOT  multispectral  image 

(a)  multispectral  image 

(b)  enhanced  image  by  merging  panchromatic  image 

(c)  Notice  the  4  aircraft,  below  the  circular  compass  swing,  near  the  top  right 
hand  comer,  enhanced  in  image  (bi  above 


I 


UNCLASSIFIED 


55 


II? 


UNCLASSIREO 


ERL-0694-RR 


J 


7  CONCLUSIONS  ^ 

Some  new  capabilities  of  commercial  satellite  images  have  been  demonstrated  in  this  report.  4ri 

A  particular  effort  has  been  made  to  use  the  images  for  quantitative  purposes. 


«  It  is  shown  that  heights  of  extended  targets  can  be  determined  to  sub-pixel  accuracy 
(2  to  3  m  in  our  case)  using  SPOT  multispectral  and  panchromatic  images  of  20  and 
10  m  pixels  respectively.  The  high  accuracy  is  achieved  by  developing  a  procedure 
for  optimal  segmentation  of  the  shadows.  The  same  procedure  may  be  extended  to 
determine  other  dimensions  of  the  targets,  such  as  the  widths  of  roads  or  the  lengths 
of  buildings. 

«  The  conventional  statistical  pattern  recognition  techniques  and  neural  net 
approaches  are  used  for  object  classification.  An  accuracy  of  the  order  of  87  % 
achieved  in  classifying  artificial  objects. 

♦  New  techniques  are  demonstrated  for  image  resolution  enhancement  in  computer 
displays.  It  is  shown  that  an  image  whose  resolution  is  degraded  by  a  magnitude  of 
up  to  8  can  be  smooth  zoomed,  using  cubic  convolution  resampling,  to  achieve  a  look 
similar  to  the  origmal.  Smooth  zooming  can  be  of  great  assistance  in  target  detection. 

report  clearly  demonstrates  the  ability  of  multispectral  satellite  images  in  enhancing 
detecting  targets  of  defence  significaitce.  Objects  such  as  aircraft  can  be  clearly  detected 
different  types  of  buildings  can  be  discriminated.  It  is  also  possible  to  determine  heights 
or  other  dimensions  of  targets,  larger  than  the  pixel  size,  to  sub-pixel  accuracy.  As  mentioned 
in  the  introduction,  the  applications  of  commocial  satellite  images  are  very  much  dependent 
on  the  data  analysing  expertise.  In  view  of  this  there  is  a  need  to  re-assess  the  capabilities  of 
civilian  satellite  images  in  surveillance  and  intelligence  gathering. 
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APPENDDCI 

TABLE  L 1  GROUND  RESOLUTION  REQUIREMENTS  FOR 

INTERPRETATION  OF  IMAGERY 

General 

Precise 

Tech 

Target* 

Detection^ 

IDC 

IDd 

Description* 

analysis^ 

Bridges 

Communications 

6 

43 

1.5 

1 

0.3 

Radar 

3 

1 

0.3 

0.15 

0.015 

Radio 

3 

1.5 

0.3 

0.15 

0.015 

Supply  dumps 

Troop  units  (in  bivouac  or 

1.5-3 

0.6 

0.3 

0.03 

0.03 

onroad) 

6 

2 

1.2 

03 

0.15 

Airfield  facilities 

6 

43 

3 

0.3 

0.15 

Rockets  and  artillery 

1 

0.6 

0.15 

0.05 

0.045 

Aircraft 

4.5 

1.5 

1 

0.15 

0.045 

Command  arvd  Control  HQ 

3 

1.5 

1 

0.15 

0.09 

Missile  sites  (SSM/SAM) 

3 

13 

0.6 

0.3 

0.045 

Nuclear  weapons  comportents 

2.5 

1.5 

0.3 

0.03 

0.015 

Vehicles 

0.6 

0.3 

0.06 

0.045 

Surface  ships 

7.5-15 

43 

0.6 

0.3 

0.045 

Land  mines 

3-9 

6 

1 

0.03 

0.09 

Ports  and  harbours 

30 

15 

6 

3 

03 

Coasts,  landing  beaches 

15-30 

43 

3 

13 

0.15 

Railroad  yards  and  shops 

15-30 

15 

6 

1.5 

0.4 

Roads 

6-9 

6 

1.8 

0.6 

0.4 

Urban  areas 

60 

30 

3 

3 

0.75 

Terrain 

90 

4.5 

1.5 

0.75 

Surfaced  submarines 

7.5-30 

4.5-6 

1.5 

1 

0.03 

Notice  that  only  the  targets  within  the  box  have  any  possibility  of  detection  using  Landsat 
TM  or  SPOT  images.  Only  very  large  targets  can  be  identified  in  general. 

^  Chart  indicates  minimum  resolution  in  meters  at  which  target  can  be  detected,  identified, 
described,  or  analysed.  No  source  specifies  which  definition  of  resolution  (pixel-size  or 
white-dot)  is  used,  but  the  chart  is  internally  consistent. 

^  Detection:  location  of  a  class  of  units,  object,  or  activity  of  military  unit 

^  General  identification:  determination  of  general  target  type. 

^  Precise  identification:  discrimination  within  target  type  of  known  types. 

^  Descriptitm:  size/dimension,  configuration/layout,  component  construction,  equipment 
count,  etc. 

^  Technical  analysis:  detailed  analysis  of  specific  equipment 
Adapted  from  Richelson,1990 
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TABLE  1.2  TARGET  DETECTION  CAPABILITIES  OF  COMMERCIAL 

SATELLITE  IMAGES 


Target* 

Detection^ 

General 

Identification^ 

Quantitative' 

Bridges 

MSS/TM 

TM/XS 

XS/P 

Roads 

MSS 

MSS 

TM/XS 

Radars 

P 

P 

Railroads 

MSS 

P 

Supply  dumps 

MSS 

P 

P 

Major  headquarters 

MSS 

TM/P 

P 

Airfield  facilities 

MSS 

TM 

P 

Aircraft 

P 

P 

P 

Rockets  and  artillery 

MSS/TM 

XS/P 

Missile  sites  (SAM) 

MSS 

MSS/TM 

P 

Surface  ships 

XS 

XS 

XS/P 

Surfaced  submarines 
Vehicles 

TM(?) 

P(?) 

XS/P 

P(?) 

Compare  this  table  with  Table  I.l 

Notice  that  the  capabilities  of  the  sensors  are  much  more  than  earlier  believed. 
Abbreviations: 

MSS:  LANDSAT  multi-spectral  scanrter;  80  meter  resolution 
TM:  LANDSAT  thematic  mapper;  30  meter  resolution 
XS:  SPOT  extended  spectrum  sensor;  20  meter  resolution 
P:  SPOT  paiKhromatic  seirsor;  10  meter  resolution 

^  No  attempt  made  to  list  all  targets  in  original  chart 

^  A  target  of  the  given  type  is  clearly  present,  but  no  details  are  apparent 

^  Class  of  bridge,  number  of  buildings,  etc,  can  be  discerned.  There  is  little  or  no  doubt 
that  the  target  has  been  properly  classified. 

^  Quantitative  measurements  of  the  target  can  be  made;  aircraft  classified  as  to 
mission  or  type.  Known  types  can  be  recognised  using  tables,  silhouettes,  etc. 
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APPENDIX  II 

PIXELISATION  EFFECT  ON  SHADOW  WIDTH  MEASUREMENT 

In  analytical  tenns  the  determination  o(  width  of  a  linear  object  can  be  achieved  in  many 
ways.  One  simple  approach  is  to  divide  the  area  covered  by  the  object  by  its  length.  This 
approach  can  not  be  direcdy  implemented  in  image  processing  for  computing  shadow  widths. 

In  Figure  11.1  shadows  are  drawn  as  the  area  between  pairs  of  lines.  Widths  are  indicated 
opposite  to  the  pairs  of  lines.  The  surface  is  divided  into  IQxlO  m  grids  to  simulate  SPOT 
image  pixels.  The  black  pixels  are  those  in  which  50%  or  more  of  the  pixel  area  is  covered  by 
shadow.  It  is  assumed  that  a  perfwt  technique  is  available  to  threshold  such  pixels  and  label 
them  as  shadows. 

Due  to  the  discrete  pixel  sizes  involved  in  images  tlw  shadow  zones  may  appear  as  continuous 
areas  or  as  fragments.  Under  perfect  thresholding  conditions,  as  described  above,  one  can 
expect  continuous  shadow  zones  only  if  the  shadows  have  widths  equal  to  or  greater  than  the 
pixel  dinwnsions. 

From  the  figure  it  is  also  clear  that  the  shadow  width  computed  by  dividing  the  area  of 
shadow  pixels  by  the  length  needs  to  be  corrected.  For  example,  the  area  covered  by  the  10  m. 
wide  shadow  is  20  pixels  or  2000  m^.  The  length  of  the  shadow  is  208.8  m.  The  computed 
width  of  the  shadow  is  938,  which  is  less  than  10  m.  In  order  to  get  the  correct  estimate  the 
length  must  be  measured  along  the  scan  line  or  the  actual  length  must  be  multiplied  by  the 
cosine  of  the  angle  between  the  shadow  zone  and  the  scan  line. 
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